Method of signal reconstruction, imaging device and computer program product

ABSTRACT

A dynamic range control is of particular interest for scenes with a high contrast between dark and bright parts. Both parts may contain detailed information, although in most cases the dark part is given priority during signal reconstruction processing. In such a case the dark parts of a scene are amplified to a level that offers sufficient visible details, whereas in most prior art cases the bright parts may exceed the maximum permissible signal amplitude and will then be clipped. Such a measure will, in most cases, cause the loss of all details above and beyond the maximum permissible signal amplitude level. It is proposed that in particular the bright parts of a scene are compressed by means of a non-linear transfer function such that the specific demands of an input signal are taken into account.

The invention relates to the method of signal reconstruction comprisinga dynamic range control processing of an input image signal to generatean output image signal. The invention also relates to an imaging devicefor a signal reconstruction comprising a means for dynamic range controlprocessing of an input image signal to generate an output image signal.Further the invention also relates to a computer program product.

An imaging device usually comprises an optical system for generating animage and a sensor means for transforming the optical image into ananalog signal. The analog signal comprises the image information. Thesensor means may either be a black/white sensor or a color sensor. Sucha sensor is usually constituted by a matrix of pixels arranged in anarray which may function as a CMOS-based device or as a CCD-type device.The analog signal of such a device comprises information according tothe optical information as sensed by each pixel and is usually convertedfor further processing by an analog-to-digital converter (ADC).

A color signal may be provided in one of the standards known as the Y-UVsystem or the RGB-system. The luminance and color coordinates of bothsystems may be transformed into each other by a suitable matrixtransformation. In the RGB-system the luminance may be derived from theR, G and B-components whereas in the Y-UV system the luminance isprovided as the Y-component.

The analog signal is transformed into a digital signal by ananalog-to-digital converter (ADC). The analog and digital informationmay be scaled in a certain bit range depending on the ADC. This range isreferred to as the dynamic range of the image. Some prior art methods,such as the one disclosed in U.S. 2001/0005227 A1, provide a suitableimaging device capable of remarkably increasing the dynamic range of anamplification-type CMOS image sensor and of obtaining a good image froma small signal to large signal amplification and preventing the signalfrom being clipped.

More advantageous contemporary methods of analog-to-digital conversionof the analog to the digital signal, such as the one disclosed by WO99/60524, attempt to increase the contrast in the resulting imagewithout necessitating an increase in the dynamic range of theanalog-to-digital converter used in converting the analog image signalinto digital information. A dynamic range of images may be enhancedwithout increasing the range by compressing the input range of the inputsignal in a smaller bit range of an output range of the output signalduring digital signal processing. Such compression of the input signalmay be advantageously performed by any desired transfer function duringdigital signal processing.

However, the provision of a suitable non-linear transfer characteristicthat can be used as a transfer function and is capable of compressingthe input signal within a dynamic range control at the processing modulecauses particular problems.

For instance, the amount of dynamic range compression itself may bespecified by an auto exposure unit in combination with a peakwhitedetector for sensoring the peak value of a white scene of an image. Thisallows the amount of dynamic range compression to be determined.However, in most cases, rather arbitrary concepts have been appliedsubsequent to the processing of a dynamic range control. This oftenresults in rather poor image quality during image amplification, as sofar it has not been possible to specifically adapt a dynamic rangecontrol processing to an input image signal. Dynamic range control is ofparticular interest for scenes with a very high contrast between thedark and bright parts. Both parts may contain detailed information but,in most cases, the dark part is given priority in contemporary devices.This often results in problems as the dark parts of the scene areamplified to a level that offers sufficient visible details, whereas thebright parts in such a case exceed the maximum permissible signalamplitude and will be clipped. This will usually cause a loss of alldetails above and beyond the maximum level of signal amplitude. Aconcept which is able to provide a specific particularly suitabletransfer function would be advantageous as this would allow to adapt theway of controlling the dynamic range of processing with respect to aspecific quality and kind of image signal.

This is where the invention comes in, the objective of which is tospecify a method and apparatus of signal reconstruction, comprising adynamic range control processing of an input image signal to generate anoutput image signal based on a concept which is able to account forspecific input image signal demands.

As regards the method, the object is achieved by a method as mentionedin the introduction, the method comprising the steps of:

-   -   providing the input signal;    -   determining an amount by:        -   specifying an input range of the input signal, and        -   specifying an output range of the output signal;    -   selecting a convex function as a non-linear transfer        characteristic capable of compressing the input signal according        to the amount of dynamic range control processing;    -   processing the input signal wherein the input signal is        transferred by means of the convex function;    -   generating the output signal as a result of the processing.

As regards the apparatus, the object is achieved by an imaging device asmentioned in the introduction, wherein according to the invention thedevice comprises:

-   -   an input means for providing an input signal;    -   a means for determining an amount comprising:        -   a means for specifying an input range of the input signal,            and        -   a means for specifying an output range of the output signal;    -   a computing means for selecting a convex function as a        non-linear transfer characteristic capable of compressing the        input signal according to the amount of dynamic range control        processing;    -   a processing means for transferring the input signal by means of        the convex function;    -   an output means for generating the output signal as received by        the processing means.

Further, the invention leads to a computer program product that can bestored on a medium that can be read by a computer system, comprising asoftware code section which induces the computer system to execute theproposed method when the product is executed on the computer system.

The proposed concept has arisen from the desire to specify anadvantageous way of controlling a signal transfer in a dynamic range bysuitably processing an image signal during signal reconstruction. Theinvention has realized that conventionally any kind of transfer functionis considered suitable to process an image signal during dynamic rangecontrol, as e.g. mentioned in WO 99/62524. However, such a generalapproach does not account for certain specifications, which maycharacterize a specific image. The main idea behind the proposed conceptis to provide a transfer characteristic capable of compressing the inputsignal, which may also be adapted to the specific demands of an inputimage to be processed. According to the proposed concept, a convexfunction is selected as a non-linear transfer characteristic capable ofcompressing the input signal according to the determined amount ofdynamic range control processing. The input signal is processed, whereinthe input signal is transferred by means of the convex functionaccording to the determined amount of dynamic range control processing.Thus, an output signal is generated wherein all details, particularlydetails of light parts as compared to dark parts, are particularly wellvisible. Information, that would be lost with conventional methods, isadvantageously preserved, albeit with an unavoidable reduction of themodulation depth.

Such an advantage is achieved by determining the amount of dynamic rangecontrol processing by specifying at least an input range of the inputsignal and an output range of the output signal. Consequently, the inputsignal is transferred by means of the convex function according to thespecific demands of an input and output signal. Therefore, the bestquality for each input signal is achieved. The method may be realizedaccording to the limits of a device used for signal reconstruction.

Further developed configurations of the invention are outlined in thedependent claims.

The input and/or output range is preferably determined by means of apeak value and/or an exposure average value taken from the signal. Suchvalues may be determined by measuring and/or performing a histogramanalysis of the signal. A luminance signal is particularly suitable as asignal.

The input signal is conveniently compressed if the peak value of theinput signal exceeds the output range. It may also be desirable tocompress a mere fraction of the image e.g. the bright scene fractions ofan image.

Most preferably the convex function is selected according to thedetermined amount of dynamic range control processing. In particular,the convex function is selected depending on the input and/or outputrange. A convex function is generally curved at the top and thereforehas at least for one value a negative curvature.

In a preferred configuration the convex function is formed by at least afirst and a second part with a kneepoint as a point of intersection ofthe first and the second part. In this case, preferably the first partof the convex function has an average steepness exceeding that of thesecond part to form a convex function. The kneepoint may be defined byx- and y-coordinates, wherein the y-coordinate corresponds to thekneelevel.

The kneepoint is preferably located on the convex function at aspecified kneelevel separating the first part from the second part. Thefirst and second part of the convex function are each formed mostadvantageously by a linear function with a constant steepness. Such aconvex function configuration allows a particularly advantageousfunctional adaptation with regard to a signal. The function itself issimple enough to keep computing efforts low and is adaptable to a signalin a particular convenient way. These and other preferred configurationswill be described in the following.

In a first variant the convex function may be selected by varying thesteepness of the second part, in particular, by simultaneously keepingthe kneelevel constant.

In a second variant the convex function may be selected by varying thekneelevel of the convex function, in particular by simultaneouslykeeping the steepness of the second part constant.

In the preferred configuration the convex function is selected dependingon the amount of dynamic range control processing function, inparticular depending on the input and/or output range, wherein acombination of varying the steepness of the first variant and varyingthe kneelevel of the second variant is available.

A particularly preferred criterion for selecting the convex function isas follows: The varying of the steepness of the second part ispreferably selected if the input range of the input signal exceeds apredetermined threshold level. Also, if a chosen kneelevel exceeds theoutput range the varying of the steepness of the second part ispreferred.

The image signal may be any signal suitable for describing an image incontemporary imaging devices. The image signal in particular has anumber of components, which may include a luminance component and/or oneor more chrominance components, e.g. the image signal is a Y-UV-signalor an RGB signal. Preferably, an amount of dynamic range controlprocessing is determined on a Y-signal, in particular a Y-signal derivedfrom an R, G, and B-component or determined on at least one component ofan R, G and B-component.

The above concept may be variously implemented in a processing chain forsignal reconstruction. The input signal is preferably a digital signal,as will be explained in more detail with reference to FIG. 1 in thedetailed description.

In particular, the digital signal is received from a whitesignal-balancing module and the output signal is provided to agamma-control module. Thus, it is possible to advantageously apply anamount of compression range common to all signal components for dynamicrange control processing and/or to process the components by means of acommon convex function.

Further, the input signal may also be an analog signal, which will beexplained in more detail with reference to FIG. 6 in the detaileddescription. In such a case the input signal is received from a sensor,in particular a sensor matrix, and the output signal is provided inparticular to an analog-to-digital converter. Most preferably in such acase a specific amount of compression range is specifically applied toat least one or all of the signal components for dynamic range controlprocessing and/or each of the components is processed by transferringthe component by means of a specific convex function according to adetermined amount specific of each component. Consequently, eachcomponent is treated in a separate and specific way according to theadvantageous demands of each component. Each component may be used toselect the steepness, and/or kneelevel and/or input range. Yet, commonsignals may also be selected from in particular luminance signals.Furthermore, the steepness and/or kneelevel and/or input range may alsobe selected according to a sensor matrix and/or a temperature value foreach signal component, in particular a color component.

If the input signal is an analog signal in a further developedconfiguration the input and/or output range may also be determined froma digital signal, which will be explained in more detail with referenceto FIG. 10 in the detailed description.

It is particularly preferable to provide an exposure measurement in aloop parallel to the dynamic range control processing. Also it ispreferable to provide a white balance control in a loop in parallel withthe dynamic range control processing. In the above further developedconfiguration advantageously one single parallel loop for an exposuremeasurement is provided.

In particular for the case of the further developed configuration it isadvantageous that original data of the input signal are retrieved. Asthe original data are most reliable for determining the amount ofdynamic range control processing, these are preferably provided to anexposure measurement and a white balance control. Preferably theoriginal data are retrieved by means of an inverse non-linear transfercharacteristic. If, however, a histogram is used for an exposuremeasurement, it is also possible to apply a histogram stretcheralternatively or additionally.

An exposure measurement is preferably controlled to assign the maximumoutput signal amplitude to a peak value of white. In particular, if aninverse non-linear transfer characteristic is used, such control ispreferably provided to prevent errors at an increasing sceneillumination.

As regards the computer program product, it may comprise a module forcalculation of a dynamic look-up table for selection of a convexfunction as a non-linear transfer characteristic depending on at leastone of the parameters selected from the group consisting of: peak value,exposure average value, input range, output range, and temperaturevalue.

The computer program product may in particular comprise a module forcalculating an inverse dynamic look-up table as an inverse non-lineartransfer characteristic. In a further configuration, if the input signalis an analog signal, the computer program product may comprise a modulefor calculating a specific dynamic look-up table and a specific inversedynamic look-up table, which is specifically adapted for at least onecomponent of the input signal.

In summary a dynamic range control has been described which is ofparticular interest for scenes with a high contrast between dark andbright parts. Both kinds of parts may contain detailed information,although, in most cases, the dark part is given priority during signalreconstruction processing. In such a case the dark parts of a scene areamplified to a level that offers sufficient visible details, whereas inmost prior art cases the bright parts may exceed the maximum permissiblesignal amplitude and will then be clipped. Such a measure will, in mostcases, cause the loss of all details above and beyond the maximumpermissible signal amplitude level. It is proposed that in particularthe bright parts of a scene are compressed by means of a non-linearfunction, so that the specific demands of an input signal are taken intoaccount. It is proposed that a scene is compressed in a preferredconfiguration, in particular the bright parts of a scene, by means of anon-linear transfer function. The transfer function is chosen to be aconvex function, which can be selected according to the demands of theamount of dynamic range control. Such a measure allows the details in abright scene part to be preserved, although this may also result in areduction of the modulation depth. Still such details are not lost butare instead preserved and remain conveniently visible. In a firstpreferred embodiment the dynamic range control processing is performedon a digital signal subsequent to a white balance control and prior to agamma-control of the camera. In such a case an analog-to-digitalconverter should provide some extra bits to enable the dynamic rangecontrol processing. In a second preferred embodiment the dynamic rangecontrol processing is performed during the early stages, i.e. “in thefront” of image processing in a camera, preferably acting on theoriginal analog signal of the image sensor. Advantageously, in such acase, an analog-to-digital converter may be applied with fewer bits thanin the first preferred configuration and a digital signal is stillconveniently quantified. For a proper color reproduction the convexfunction, as a non-linear transfer characteristic, is preferably appliedto at least one or all of the color components of an image signal. In afurther developed configuration the input signal is also an analogsignal and the output range is determined from a digital signal. Theproposed method is advantageously applied to a signal of an RGB-colorsignal of an image sensor. A computer program is specifically adaptedvia the implementation of modules to calculate specifically adaptedlook-up tables (LUTs).

Preferred embodiments of the invention will now be described in adetailed description with reference to the accompanying drawing. Thesefigures of the drawing are meant to show examples to clarify theproposed concept in connection with the detailed description of apreferred embodiment and in comparison with prior art. While theconsidered preferred embodiments of the invention will be shown anddescribed, it should of course be understood that various modificationsand changes in form or in detail could readily be made without departingfrom the spirit and scope of the invention. It is therefore intendedthat the invention may not be delimited to the exact form and detailsshown and described herein nor to anything less than the whole of theinvention disclosed herein and as claimed hereinafter. Further, thefeatures described in the description, the drawings and the claimsdisclosing the invention, may be essential for the invention consideredalone or in combination.

The drawing shows in:

FIG. 1 a first preferred embodiment of the method of signalreconstruction, wherein an automatic exposure measurement and a dynamicrange control are applied to a digital signal behind ananalog-to-digital converter and subsequent to a matrix module and awhite balance module;

FIG. 2 a preferred scheme for selecting the convex function as anon-linear transfer characteristic;

FIG. 3 a first preferred embodiment of the convex function having afixed kneelevel and a variable compression in a second part of theconvex function;

FIG. 4 a second preferred embodiment of the convex function having afixed compression in the second part of the convex function and avariable kneelevel;

FIG. 5 an exemplifying embodiment of the convex function, wherein theparameters of a module for calculation of a kneelevel are defined;

FIG. 6 a second preferred embodiment of the method of signalreconstruction, wherein an automatic exposure control and a dynamicrange control are applied to an analog signal of an image sensor beforean analog-to-digital converter is applied;

FIG. 7 a schematic view of a set of specific knee transfer functions,each of which is used respectively as a convex non-linear transfercharacteristic for each of the color components of an image signalprocessed according to the second preferred embodiment of the method ofsignal reconstruction;

FIG. 8 calculated versions of the convex functions as have been shown inprinciple in FIG. 7, to be used as an adaptation of a matrix forobtaining a better quantification;

FIG. 9 a flow diagram illustrating the processing and selection of aconvex function according to the second preferred embodiment with regardto the parameters “kneelevel” and “peak value”;

FIG. 10 a third preferred embodiment of the method of signalreconstruction similar to the one shown in FIGS. 6 to 8, wherein dynamicrange control processing is applied to an analog signal and an automaticexposure control is applied to a digital signal;

FIG. 11 a schematic view of an example of an inverse dynamic look-uptable as calculated by a respective software code section;

FIG. 12 some exemplifying histograms of a picture with different sceneilluminations in the range from 100% to 40%;

FIG. 13 like FIG. 12 for different scene illuminations in the range from40% to 100%;

FIG. 14 a simplified RGB reconstruction in even rows at half the sensorpixel clock to be used within the second or third preferred embodimentof the method as shown in FIGS. 6 and 10 respectively;

FIG. 15 a scheme of automatic exposure measurement generating acontinuous RGB-measurement signal in even rows to be used for RGBreconstruction of FIG. 14;

FIG. 16 a further scheme of automatic exposure measurement generating acontinuous RGB-measurement signal applicable at a quarter of the sensorclock speed.

The following detailed description accompanies the drawing and comprisesthe following chapters:

-   1. Dynamic range control subsequent to a matrix and white balance    control    -   1.1 Two types of transfer characteristics for a dynamic range        control-   2. Dynamic range control before an analog digital converter    -   2.1. Dynamic range control with a parallel processing loop for        measurement    -   2.1.1. The influence of a matrix and white balance parameters on        a knee transfer    -   2.1.2 Calculation of a dynamic look-up table for an RGB-sensor        signal    -   2.2. Dynamic range control with an inverse dynamic look-up table        for measurement    -   2.2.1 A problem with increasing scene illumination

Appendix: A simplified RGB reconstruction of a dynamic range controlapplied to an analog sensor signal.

1. Dynamic Range Control Subsequent to a Matrix and White BalanceControl.

FIG. 1 shows the block diagram of a scheme of signal reconstruction,comprising a dynamic range control (DRC) located between an AWB control(Auto White Balance) and a gamma processing.

An image sensor with an RGB Bayer color array, is followed by a 12-bitADC (analog-to-digital converter). The 12-bit ADC is of coursearbitrary. Depending on the application it can be any converter betweena 10-bit and a 16-bit converter, wherein it is assumed that 2 or 3 bitsare reserved for the dynamic range control.

The proposed method of signal reconstruction comprising a dynamic rangecontrol processing of an image is preferably applied with images, suchas e.g. computer pictures, having a 10 to 16 bit depth for each color.On 8-bit or lower depth computer pictures it may be applied as well,although then there is the risk of visible quantification.

In the preferred embodiment a 12-bit ADC with 2 bits for dynamic rangecontrol has been selected. A 100% signal amplitude is achieved with 10bits. This allows a maximum over-exposure of a factor of 4, whichcorresponds to a signal amplitude of 400% or 12 bits.

Following the 12-bit ADC a multiplexed digital RGB signal is availablein the form of a row alternating RG and GB sequence due to the Bayercolor array. Following the RGB reconstruction three continuous RGBsignals are available, each with a 12-bit quantification.

The color correction by means of the sensor matrix and the AWB controlis followed by an auto exposure (AE) measurement in a parallel loop.This AE unit determines and controls the exposure time of the imagesensor and also predicts the DRC parameters. For the sake of clarity, itshould be mentioned that the AE control is best executed in a closedloop, while the DRC is advantageously a predictive controller.

From the ADC to the DRC a 36-bit quantified RGB signal is applied, 12bits for each primary color. After the DRC the RGB data consist of only10 bits per color (30 bits for RGB), corresponding to a 100% signalamplitude. FIG. 3 exemplifies a dynamic range compression of 4 times.

In the block diagram of FIG. 1 it is assumed that the AE measurement isexecuted on a luminance Y-signal, of which the arbitrary RGB weights arechosen according to the color television transmission agreement:Y=0.3*R+0.59*G+0.11*B.

The RGB weights in the luminance signal are usually derived from theluminance contribution of the early CRT phosphors used in the NTSCtelevision system. Today the luminance output of the phosphors has beengreatly improved, causing a completely different luminance contribution(Y=0.22R+0.71G+0.07B), as well as another color gamut. For all videocameras of known art, including the NTSC countries such as the USA andJapan, the color gamut has been adapted to the new CRT phosphors. As aresult, the old luminance weights only concern an appointment about thetransmission of the television signals. Moreover, due to the matching ofthe camera and CRT color gamut, they do not influence the colorreproduction at all.

Behind the white balance control processing the RGB signals are supposedto be equal in the case of white colors. This means that the samedynamic range transfer can be applied advantageously to each of thethree RGB signals. Similarly, the same gamma-transfer can be applied. Ifa look-up table (LUT) is used, one single LUT is sufficient for the DRC.Look-up tables will be described in further detail below.

There are many ways of realizing the AE control and of determining theamount of dynamic compression. As neither AE control nor the measuringof the dynamic compression are main subjects of this report, one canassume that the average signal of the whole scene is used for the AEcontrol and that a rather arbitrary peakwhite detector is used todetermine the dynamic compression. In this chapter a compression of fourtimes is assumed (4096/1024). Before the DRC this results in a maximumpeakwhite amplitude of (2¹²−1)=4095. RGB input signals to the DRC largerthan 4095, which can be generated simply and exclusively by the matrixand the AWB control, will be limited to a maximum output level of(2¹⁰−1)=1023. The 12-bit ADC has already limited the RGB sensor signalto a maximum value of 4095. As it is rather unlikely that the RGBreconstruction will add artifacts larger than that value, the matrix andthe AWB control are the only ones left that may cause artifacts.

1.1 Two Types of Transfer Characteristics for a Dynamic Range Control.

A proper choice of a kneelevel is illustrated in FIG. 2. The kneepointcan be regarded as the point at which the dynamic compression starts. Ingeneral this is rather arbitrary and will be discussed further in thischapter.

In common practice dynamic range control (DRC) is often called kneecontrol. Therefore, in addition to the peakwhite parameter, the DRCparameters contain the word knee, e.g. kneelevel and kneecompression.The amount of compression is defined as:kneecompression=(maximum output level−kneelevel)/(peakwhite−kneelevel)

The maximum output level according to FIG. 2 is 1023, which correspondsto an output signal of 10 bits.

There are two types of particularly advantageous knee transfers. Thesehave been referred to as the first variant and the second variant in thegeneral part of this application and are referred to here as kneetype 1and kneetype 2. The first kneetype assumes a fixed kneelevel, so theattenuation above the kneelevel will vary as a function of the amount ofcompression as shown in FIG. 3. When considering the performance ofcompressed pictures, it is very disadvantageous if steep curves of asmall dynamic compression factor are used, especially because mostscenes need only a relatively small amount of compression.

The second kneetype supposes a fixed attenuation and, as a consequence,a varying kneelevel, of which an example is shown in FIG. 4. From apicture performance point of view this kneetype has some advantages atsmall dynamic compression factors, covering most of the scenes inpractice. At high compression factors, however, the first kneetype withits fixed kneelevel is more advantageous. Both types of knee transfermay be combined. Either one of them may be advantageously selecteddepending on the suitability of the parameter.

The combination of both kneetypes offers the best performance and hasbeen applied to the calculation of the dynamic range control in thefollowing software description.

{Declaration of Variables, See Also FIG. 5}   peakwhite, { peakwhitewithout dynamic range compression }   kneetype, { kneetype 1 with fixedkneelevel, type 2 with fixed compression }   kneelevel, { preferredkneelevel }   newkneelevel, { really applied kneelevel }  refkneecompres, { preferred amount of compression }   kneecompres, {actually applied compression }   zerointersection { intersection ofcompressed line for Yin=0 } { Calculate newkneelevel as function ofkneetype }   if peakwhite>4095 then peakwhite=4095   newkneelevel=1023  if peakwhite>1023 then { dynamic compression is desired }   begin {default kneetype = 2, with a fixed kneecompression, so}  kneecompres=refkneecompres { find zero_intersection (Yin=0) for liney2 for which counts: y2 = zero_intersection + kneecompres*newkneelevel(in Yin direction) for peakwhite for the y2-line counts: 1023 =zero_intersection + kneecompres*peakwhite, so}  zero_intersection=1023−(kneecompres*peakwhite) { find newkneelevel atthe intersection of the lines y1 and y2, y1 = 1.0*newkneeleve y2 =zero_intersection+kneecpmpres*newkneelevel }   if (1.0−kneecompres)<>0then  { prevent division by zero }  newkneelevel=zero_intersection/(1.0−kneecompres)   elsenewkneelevel=1023   if newkneelevel<kneelevel then { step over tokneetype=1 }   begin   newkneelevel=kneelevel  {maintain kneelevel, findkneecompres value}  kneecompres=(1023−newkneelevel)/(peakwhite−newkneelevel)   end end2. Dynamic Range Control Before an ADC.

The application of a dynamic range controller before the ADC becomesnecessary as the state-of-the-art IC technology is not yet able to offeran ADC with sufficient bits as shown in FIG. 1. This may be the case ifthe ADC has to be integrated on the (CMOS) image sensor or on thesignal-processing chip. It is expected that with the further refinementof IC technology it will only be a matter of time before both optionscan be realized. Nevertheless, two methods of a DRC acting in advance ofthe ADC, i.e. in the analog signal domain, will be considered here. Bothmethods, just like the one described in chapter 1, will predict theamount of dynamic range compression as a function of the AE control andthe detected peakwhite. The first preferred embodiment using an analogsignal uses an independent, parallel measurement circuit. The secondpreferred embodiment using an analog signal performs the measurement viathe non-linear DRC and uses an inverse knee transfer subsequent to thematrix and AWB control in order to again retrieve the ‘original’ datafor AE control and peakwhite detection. The first embodiment forprocessing an analog signal is described in chapter 2.1. The secondembodiment for processing an analog signal is described in chapter 2.2.

2.1 Dynamic Range Control with a Parallel Processing Loop forMeasurement.

FIG. 6 shows a DRC block diagram with a parallel processing and AE loop,which is independent of the non-linear DRC because it uses the linearsensor signal. The amount of dynamic range control is predicted via thisAE loop. Of course the AE measurement can be completely realized in theanalog signal domain, or possibly on the sensor itself, just as in thecase of the DRC and 10-bit ADC. Here, however, a simplified digital AEloop is shown (that can also be implemented on the sensor).

This digital measurement loop starts with an ADC of only 8 bits, whichappears to be sufficient for measurement purposes and has been proven bycomputer simulations. The multiplexed RGB sensor signal is thentransferred to three continuous RGB signals (‘RGB pixel’ in the diagram)by combining the pixels within a 2×2 array, an example of which is givenin the appendix. After the simplified RGB signal reconstruction the samematrix and AWB control as in the upper and true signal path are applied.The only difference is in the 8-bit signal handling. The RGB signals arethen offered to the AE measurement circuit. For the true signal path a10-bit ADC is applied after the analog DRC. The quantification closebefore the gamma circuit is the same as for the block diagram of FIG. 1.

The RGB signals following the white balance control should be equal forgray or white colors. Moving backwards from the AWB control, via thematrix towards the analog DRC, it will become clear that it is veryunlikely that the three RGB signals are still equal for the white colorafter the AWB control. This will be the case if, for example, the colortemperature of the scene corresponds to 6500 K and the matrix is theunity matrix. Thus, 3 knees usually have to be provided in this firstembodiment for the processing of an analog signal.

2.1.1 The Influence of a Matrix and White Balance Parameters on the KneeTransfer.

The sensor matrix uses the a_(xx) parameters as follows: It is essentialthat the multiplication of the white balance parameters with the sum ofmatrix parameters be in unity. Assuming the following sensor matrix

-   -   a11 a12 a13    -   a21 a22 a23    -   a31 a32 a33        and the measured white balance parameters awbR and awbB are        given. In such a case equal analog knee transfers in the front        are only obtained if:    -   (a11+a12+a13)*awbR=1    -   (a21+a22+a23)=1    -   (a31+a32+a33)*awbB=1        In such case the inverse b_(xx) matrix is defined as:    -   b11 b12 b13    -   b21 b22 b23    -   b31 b32 b33        It holds that A×B=1;        where 1 is the unity matrix.

The awbR and awbB parameters are the measured white balance parameterswhen an arbitrary scene color temperature is given. According to theWorld Gray Assumption Method (WGA) the following holds true:

-   -   awbR=totalGreen/totalRed    -   awbB=totalGreen/totalBlue,        where totalRed, totalGreen and totalBlue represent the total of        the RGB color amplitudes measured over the whole scene. Just as        in the case of the inverse matrix, the inverse white balance        parameters are also needed in order to find the knee transfer        for the analog DRC in the front for each primary color. This        requires a great deal of calculation power because the so-called        ΣXiwb-parameters need to be calculated first, followed by the        RGB-transfer curves. Abbreviations are used: Σ=sigma and X=the        R, G or B primary color.)        ΣRiwb=(1/awbR)*b11+b12+(1/awbB)*b13        ΣGiwb=(1/awbR)*b21+b22+(1/awbB)*b23        ΣBiwb=(1/awbR)*b31+b32+(1/awbB)*b33  [1]

FIG. 7 gives an example of three different knee transfers for the analogDRC in the front. The applied matrix is in unity and the scene colortemperature is about 4000K (Kelvin). It is evident that the outputsignal of the red knee curve exceeds the maximum value of the 10-bit ADCby a factor of 1.22. This means that an 11-bit ADC should be applied or,in case of maintenance of the 10-bit version, that the maximum outputlevel should be lowered to 2⁹−1=511, so that 1 bit is again availablefor the red or blue curve as a function of a lower or higher scene colortemperature than the white of an average daylight of 6500K.

In case of a unity matrix, the inverse matrix is unity as well. TheΣXiwb-parameters are then determined by the white balance parametersonly.ΣRiwb=1/awbRΣGiwb=1.0ΣBiwb=1/awbB  [2]A black body radiator of 3200K gives the following ratio for the primarycolors:

-   -   R:G:B=1.45:1.00:0.37        In order to achieve R=G=B after the white balance control, the        white balance parameters have to be:    -   awbR=1/1.45 and awbB=1/0.37        As a consequence:    -   ΣRiwb=1.45, ΣGiwb=1.0 and ΣBiwb=0.37

The maximum RGB outputs of the knee transfer will then be 1.45, 1.0 and0.37 times the maximum output of 1023 respectively.

For a color temperature of 30,000K the following holds true:

-   -   R:G:B=0.85:1.00:1.83

Here the maximum output of the blue color after the knee transfer willbe 1.83 times the maximum output of 1023. Thus, in the case of a unitymatrix, the factor to increase the signal amplitude of the ADC by meansof a single extra bit will be sufficient for a color temperature rangevarying from 3200K to 30,000K. If an extra bit is assumed for the ADC,i.e. a total of 11 bits, the maximum output value will be 2¹¹−1=2047. Inpractice, white balance circuits will start to limit the red and bluegain factors towards rather low (3200K) and high (30,000K) colortemperatures in order to maintain something of the color sphere of theoriginal scene. Thus, the increase of the red and blue amplitude will besomewhat smaller than 1.45 and 1.83 respectively.

The maximum output after the matrix and AWB control as shown in FIG. 6remains, 1023, however, because the RGB-amplitudes are equaled out forwhite. It is also important to be aware that the knee transfer of greenin FIG. 7 corresponds to the transfer of the DRC after the matrix andAWB control as described in chapter 1.

In case of a color temperature of 6500K, for which the white balanceparameters awbR and awbB are in unity, a formula can be written, whereinthe sum of the inverse matrix parameters determines whether the maximumADC value of 2047 will be superseded. This specific case can beimportant for a possible adaptation of the matrix and will be used inthe following explanation.

For a color temperature of 6500K for the ΣXiwb-parameters count:ΣRiwb=b11+b12+b13ΣGiwb=b21+b22+b23ΣBiwb=b31+b32+b33  [3]

In order to remain within the 11-bit range of the ADC it may benecessary to resize the matrix. For this purpose, and using a formula,[1] the ΣXiwb-values should be calculated for the limits of the colortemperature range, assumed to be 3200K and 30,000K in this case. Thelargest of the ΣXiwb-values should then be taken. If one of them islarger than two, it should be lowered to just below two by making aproportional adjustment to the whole matrix. This will guarantee thatthe maximum output value of 2047 will not be exceeded. Conversely, iffor 6500K the ΣGiwb-value (formula [3]) is smaller than one, then thewhole matrix should be proportionally increased in such a way that theΣGiwb-value becomes one. This will guarantee a better quantification ofthe sensor signal. The first priority, however, is given to resizing thematrix as a function of the limits of the color temperature range.

Two examples of existing matrices will be given to clarify thisproportional matrix adjustment.

FIRST EXAMPLE

Matrix 1 (an FT matrix) 3200K 6500K 30,000K 2.000 −0.771 0.006 ΣRiwb =1.560 ΣRiwb = 1.454 ΣRiwb = 1.540 −0.238 0.762 −0.291 ΣGiwb = 2.227ΣGiwb = 2.490 ΣGiwb = 2.922 0.045 −0384 0.915 ΣBiwb = 1.256 ΣBiwb =2.066 ΣBiwb = 3.155

ΣBiwb at 30,000K is much larger than 2 and will be adjusted to 1.99,resulting in the following matrix and the corresponding inverse matrix:3.171 1.222 0.009 0.363 0.422 0.132 −0.377 1.240 −0.461 0.123 1.0990.349 0.071 −0.609 1.451 0.034 0.440 0.829

If the gain of the original matrix had been smaller, this would havegiven the same result. By re-adjusting all matrix parameters by a factorof 3.171/2.000=1.5855, the auto exposure gain will also, due to theclosed AE loop, be automatically adapted by the inverse gain factor usedfor the matrix. If, for example, the original AE gain is 2.27 for aparticular scene, it will become 3.60 after the re-adjustment of thematrix. The total gain of the AE loop for that scene will thus bemaintained.

SECOND EXAMPLE

Matrix 2 (a CMOS matrix) 3200K 6500K 30,000K 1.760 −0.599 0.415 ΣRiwb =1.010 ΣRiwb = 0.694 ΣRiwb = 0.539 −0.460 1.787 −0.130 ΣGiwb = 0.852ΣGiwb = 0.781 ΣGiwb = 0.760 −0.469 −0.496 2.908 ΣBiwb = 0.441 ΣBiwb =0.594 ΣBiwb = 0.851

None of the ΣXiwb-values exceeds the factor of two. The ΣGiwb-value at6500K is smaller than one and will be adjusted to 1.0. This will resultin the following matrix and, after an extra check, in the followingΣXiwb-values for the color temperature limits: 3200K 6500K 30,000K 1.375−0.468 0.324 ΣRiwb = 1.293 ΣRiwb = 0.888 ΣRiwb = 0.670 −0.359 1.396−0.103 ΣGiwb = 0.935 ΣGiwb = 1.000 ΣGiwb = 0.973 −0.362 −0.388 2.272ΣBiwb = 1.503 ΣBiwb = 0.760 ΣBiwb = 1.089

The extra check clarifies that none of the ΣXiwb-values exceeds thefactor of two. There are however matrices where this happens. In such acase another adjustment is required. The inverse matrix is shown below:0.759 0.227 −0.098 0.207 0.787 0.006 0.163 0.172 0.424

FIG. 8 shows the results of the knee transfers after the adjustment ofmatrix 2. The gain of the original was too large. The resized matrixoffers knee transfers, especially green, on or close to the maximum RGBoutput of 1023 and, as a consequence, a better quantification.

In FIGS. 7 and 8 kneetype=2 has been applied to the different kneetransfers. A slightly better color performance occurs for kneetype=2(with a fixed compression) than for kneetype=1 (with a fixed kneelevel).For kneetype=2 the result of the processed picture is the same as in thecase of the knee processing after the matrix and AWB control, asdescribed in chapter 1. Kneetype=1 shows a small color and amplitudedeviation. Further, it is evident that neither the heaviness of thesensor matrix, nor the range of the white balance will influence theperformance of this front knee processor. For implementation it is,however, important to be aware of the range needed for the threedifferent knee transfers.

As the sensor signal is a multiplexed signal, the realization of threedifferent knee transfers requires a selection switch that controls theknee transfer for each color. A preferred way of implementation could beachieved by switching kneelevelR(G,B) and peakR(G,B) as a function ofthe actual color offered by the sensor. FIG. 9 shows an example of howthree different knee transfers can be realized by using a single ‘RGBknee transfer processor’ that receives the kneelevels and peak-settingsvia two switches in phases related to the sensor colors.

2.1.2. Calculation of the Dynamic Look-Up Tables (Dynamic Luts) for anRGB Sensor Signal.

The look-up table (lut) of the DRC, hereinafter also referred to asdynamiclut, now has to be calculated. Because this procedure also countsfor the DRC as described in chapter 1, four dynamicluts are calculated.{ Declaration of variables } EXi, { is unity for a conventional DRC,otherwise ΣXiwb for DRC in front } dynamiclut{circumflex over ( )}[k,i],{ the knee transfer for the conventional DRC (k=0) and for the front DRC(k=1 to 3), the parameter i represents the input position} peakwhite, {peakwhite without dynamic range compression } kneetype, { kneetype=0: nodynamiclut has been applied, kneetype=1 with fixed kneelevel andkneetype=2 with fixed compression } newkneelevel, { really appliedkneelevel as already calculated in chapter 1.1} kneecompres, { reallyapplied compression } { Start of the calculation of the dynamicluts } if(peakwhite>1023) and (kneetype>0) then { for peakwhite<1024 no kneetransfer is needed } for k=0 to 3 do { k=0 for conventional DRC, k=1 to3 for DRC in front } begin   case k of   0: EXi=1 { conventional DRC }  1: EXi=ERiwb   2: EXi=EGiwb   3: EXi=EBiwb   end {k case}   for i=0 toEXi*peakwhite do { also peakwhite has to be multiplied with EXi } beginif i>EXi*newkneelevel then { compressed transfer part }  j=EXi*newkneelevel+kneecompres*(i−EXi*newkneelevel)   else j=i {linear transfer part }   dynamiclut{circumflex over ( )}[k,i]=j end  for i=EXi*peakwhite+1 to 4095 do     dynamiclut{circumflex over( )}[k,i]=j { above peakwhite+1 the transfer is flat }   end   else ifkneetype=0 then   begin { no dynamiclut has been applied }     for k=0to 3 do for i=0 to 1023 do dynamiclut{circumflex over ( )}[k,i]=i    for k=0 to 3 do for i=1024 to 4095 do dynamiclut{circumflex over( )}[k,i]=255   end { for the analog DRC as described in chapter 2.2,the inverse lut will be calculated } if peakwhite>1023 thenInverseDynamicLUT { see chapter 2.2 for this procedure }

For k=0 the dynamiclut subsequent to the matrix and AWB control is theresult, an example of which is shown in FIG. 5. As has already beenexplained in chapter 1, the same knee transfer is applied to the RGBsignals.

For k=1 to 3, as a function of the inverse matrix and the inverse whitebalance parameters according to formula [1], three different kneetransfer curves will result for the RGB sensor signals in the front.FIGS. 7 and 8 show two examples of those knee transfers. As the inversesensor matrix is fixed, these analog knee transfer curves have to berecalculated every time the white balance parameters change. Only incase of an ideal unity matrix and of unity white balance parameters willthe three transfer curves in the front match the curve of the dynamiccompression as applied after the matrix and AWB control.

2.2 Dynamic Range Control with an Inverse Dynamic LUT for Measurement.

The second preferred embodiment of an analog DRC acting before the ADCwill be considered here. The block diagram of FIG. 10 shows that the AEmeasurement is executed via the processing path, thus including thenon-linear DRC in the front.

The three different knee transfers in the front will disturb the AE anddynamic range measurement after the matrix and the AWB control.Therefore, the luminance signal is processed first with an inversedynamiclut before the measurement occurs. This will undo the effect ofthe non-linear transfers in the front and will make it possible topredict what should happen there again. Due to the inverse dynamiclutthe measurement results will be the very same as those in chapters 1 and2.1. It is, however, problematic if the scene illumination isincreasing. This will be elucidated in chapter 2.2.1.

The procedure of the InverseDynamicLUT has already been mentioned in theprevious chapter. After that place the last rule of the softwaredescribing the calculation of the dynamic look-up table reads:

-   -   if peakwhite>1023 then InverseDynamicLUT

The software procedure of the InverseDynamicLUT used here is one of thepossible methods of calculation and has been realized as follows:

Procedure InverseDynamicLUT { Declaration of variables }  peakvalue, {1023 or a value between 1023 and peakwhite }  maxdynalutvalue, { themaximum value of dynamiclut{circumflex over ( )}[0,i] } begin {calculate inverse dynamiclut}  for i=0 to newkneelevel dodynamiclut{circumflex over ( )}[4,i]=i  { linear knee  tranfer }  fori=newkneelevel+1 to peakvalue do  begin { inverse part of dynamiclut[4]}   dynamiclut{circumflex over( )}[4,i]:=newkneelevel+(i−newkneelevel)/kneecompres   if i=peakvaluethen {after peakvalue maintain maxdynalutvalue }  maxdynalut=newkneelevel+(peakwhite−newkneelevel)/kneecompres  end  fori=peakvalue+1 to 4095 do   dynamiclut{circumflex over( )}[4,i]=maxdynalut end {of Procedure InverseDynamicLUT}

FIG. 11 shows an example of an inverse dynamic look-up table, thevariable dynamiclut[4] in the above software module. The conventionaldynamic look-up table, the one acting before gamma as shown in FIG. 1,is represented by the variable dynamiclut[0] in the above softwaremodule. If the compression of the variable dynamiclut[0] from thevariable newkneelevel to the variable ‘peakwhite’ is equal to thevariable ‘kneecompres’, then the amplification in the same part of theinverse variable dynamiclut[4] amounts to 1/kneecompres. For example acompression factor of 0.25 in ‘dynamiclut[0]’ results in a gain factorof 4 in ‘dynamiclut[4]’. By using the output of ‘dynamiclut[0]’ as theinput for ‘dynamiclut[4]’ a linear transfer curve up to peakwhite willagain be obtained.

As the maximum luminance output value after the matrix and AWB controlis limited to 1023 (the input is ‘dynamiclut[0]’), at a first glance itis sufficient to realize an inverse dynamic look-up table to 1023. Asthe AE control is acting in a loop the value of 1023 as the maximumluminance output could well be exceeded. Therefore, it is better toapply a ‘peakvalue’ somewhat beyond 1023, preferably between 1023 and‘peakwhite’.

FIG. 11 shows two inverse dynamic look-up table curves, one forpeakvalue=1023 and one for peakvalue=peakwhite.

It should be noticed that if a histogram has been used for the AEmeasurement, it is also possible to apply a histogram stretcher withinthe limit of the variables newkneelevel to peakwhite instead of theinverse dynamiclut as described here. The histogram stretch should beprocessed up to the limit of peakwhite in order to be able to recoverthe original histogram again.

2.1.3 A Problem with Increasing Scene Illumination.

As indicated above, the performance of using an inverse dynamic look-uptable for the DRC in the front is the same as that for the methoddescribed in the chapter using a parallel measurement circuit. Beforeshowing what may happen with a decreasing scene illumination, somevariables followed by the general procedure of the auto exposure loopwill first be elucidated. {Declaration of variables}  measuredpeakwhite,  {the measured peakwhite value of the   scene}  measured Average,{ measured average of scene }   referenceAverage, {reference average value to control to }   measuredAEgain, { the measuredauto exposure gain from scene }   AEgain, { product of AEgain andmeasuredAE gain to control   image sensor }   peakwhite  { measuredpeakwhite multiplied with   measuredAEgain }In the following, in 8 steps the general procedure for an AE controlwith the DRC in the front and an inverse dynamic look-up table isdescribed:

-   -   1. Start with an initialization: AEgain=1.00, all dynamic        look-up tables, including the inverse dynamic look-up table, are        set to linear mode.    -   2. Via the DRC in the front, the reconstruction, the matrix and        the A WB, a luminance signal is realized, the ‘measuredAverage’        and the ‘measuredpeakwhite’ values of which are measured after        stretching the luminance signal by the inverse dynamic look-up        table. The ‘measuredAverage’ and the ‘measuredpeakwhite’ values        can also be obtained from the luminance histogram of the scene.        In that case, an alternative to the inverse dynamic look-up        table can be a histogram stretcher acting from the        ‘newkneelevel’ value to ‘peakwhite’ If the histogram has been        measured via the inverse dynamic look-up table, no histogram        stretcher is of course needed.    -   3. The following parameters are then determined: measuredAEgain,        AEgain and peakwhite.    -   measuredAEgain=referenceAverage/measuredAverage        AEgain=AEgain*measuredAEgain        The auto exposure control is a closed loop of which finally the        AEgain controls the exposure time of the image sensor.        peakwhite=measuredAEgain*measuredpeakwhite    -   5. In order to prevent an error in an increasing scene        illumination, due to the inverse dynamic look-up table, the        following rule is needed:    -   if peakwhite<=1023 then AEgain=measuredAEgain*1023/peakwhite    -   6. if peakwhite>1023 then calculate newkneelevel, see chapter        1.1.    -   7. if peakwhite>1023 then calculate the dynamicluts, see chapter        2.1.2.    -   8. Next calculate the inverse dynamic look-up table, see chapter        2.2. if peakwhite>1023 then InverseDynamicLUT        Finally, the AE measurement restarts at step 2, and so on.

With the aid of FIG. 12 and the general procedure for an AE control asoutlined above, the following explains what happens if the sceneillumination decreases from about 100% to 40%. The results are drawnfrom an original figure, starting with an illumination of 100%. A colortemperature of 6500K and a unity matrix are assumed, resulting in equaldynamic RGB look-up tables in the front.

At the initialization of step 1, the AEgain=1.00 and all look-up tablesare set to linear. For all situations A to D in FIG. 12referenceAverage=512 and kneecompres=0.25 for kneetype=2. The measuredhistogram of the scene is shown at the top of FIG. 12 at A=start. Thehorizontal axis of a luminance histogram represents the signal amplitudedivided into 2^(n) segments. With a 10-bit ADC n can be chosen between 6and 10, i.e. 64 and 1024 segments. The vertical axis represents how manypixels of the total scene match the value of a horizontal gray-segment.Adding the counted values in all horizontal segments results in thetotal number of pixels of the scene. On the right hand side the measuredand calculated parameters are shown as they are after execution of theprogram steps 2 to 8. The dynamic look-up tables shown are also obtainedas they are after the execution of step 8. During step 3 the followingis calculated:measuredAEgain=512/348=1.47, AEgain=1.00*1.147=1.47 andpeakwhite=1.47*1004=1476.

In the second loop as shown in situation B of FIG. 12, steps 2 to 8 arethen repeated. The RGB dynamic look-up tables have now been activatedand the histogram has been measured via the inverse dynamic look-uptables. The AEgain, ‘peakwhite’ and ‘newkneelevel’ parameters aremaintained. Only the measured parameters have changed due to the applied‘AEgain’ of 1.47. If nothing happens to the scene illumination,situation B of FIG. 12 will be maintained during the following cycles ofthe AE measurement loop.

In situation C of FIG. 12, the scene illumination is then lowered from100% to 40%. The measured histogram will shrink 2.5 times in amplitude(being the horizontal axis). As a consequence the ‘measured Average’ and‘measuredpeakwhite’ values will also decrease by a factor of 2.5. Inorder to compensate for the illumination loss of a factor of 2.5 the‘measuredAEgain’ will increase 2.5 times and the final ‘AEgain’ willbecome 1.47*2.5=3.68.

At situation D the change in illumination has been compensated by meansof the AEgain and, in addition to that parameter, all others are thesame as for situation B.

In conclusion, for a decreasing scene illumination the method using aninverse dynamic look-up table behaves in the same way as the method inchapter 2.1 with a parallel AE measurement.

Note that step 5 has not been activated at all because ‘peakwhite’ hasbeen larger than 1023.

However, in case of an increase in the scene illumination, a problemcould occur if, for the time being, step 5 of the general AE measurementis omitted. This will be explained by increasing the illumination of theoriginal of a figure from 40% back to 100% again.

FIG. 13 starts with situation D, which is copied from situation D ofFIG. 12.

In situation E of FIG. 13 the illumination is increased to 100%. Due toAEgain still being 3.69, and the dynamic look-up table being followed bythe inverse dynamic look-up table, all luminance values above 1476 arelimited (clipped) to that value. As a lot of data is clipped, a largehistogram segment occurs near to value 1476 which corresponds to themeasured peakwhite. The measured average has become very high as well(988). Step 2 results in the following parameters:measuredAEgain=988/512=0.52, AEgain=3.69*0.52=1.92 andpeakwhite=0.52*1476=768.Steps 6, 7 and 8 of the general procedure are not activated becausepeakwhite is not larger than 1023. This means that the previous(inverse) dynamic look-up tables will be maintained.

By omitting step 5 in the general AE procedure, the intermediate state Ewill finally, i.e. after already two loops, become situation F, whichappears to be a stable situation. The dynamic look-up tables and, inconsequence, all other parameters shown on the right side of situationF, clearly differ from the desired situation B as shown in FIG. 12. Thisis caused by the fact that a part of the scene data is still clipped.Peakwhite is not the desired peakwhite value, because the last segmentof the histogram contains an undefined amount of clipped data.Therefore, a solution that indicates how much data has been clippedcannot be applied. The software simulations clarify that the omission ofthe condition that peakwhite should be larger than 1023 in step 6, 7,and 8, causes an instability of the AE control.

Of course, there may be other possible solutions as well. The oneapplied here adds step 5 to the general AE procedure. As previouslyoutlined, in situation E of FIG. 13 the steps 6, 7, and 8 are notexecuted. Because peakwhite is smaller than 1023 step 5 will becomeactive:AEgain=measuredAEgain*1023/peakwhite=0.52*1023/768=0.69.

With step 5 activated all parameters in situation E are the same as ifstep 5 had been omitted. The only difference concerns the ‘AEgain’ whichis 1.33. In the next loop the desired dynamic look-up tables havealready been found and are the following loop situation Fp isillustrated in FIG. 13 with its stretched histogram. As can be seen,situation Fp is very similar to situation B in FIG. 12.

Finally the following has to be noted:

1. The use of step 5 in the loop has a very interesting advantage forthe AE control. If for example a text on a white paper is measuredwithout step 5 having been activated then the AEgain will becomesomewhat larger than 0.5. The signal amplitude corresponding to thewhite paper will become about 50%, and will thus be displayed as a grayinstead of a white paper. With step 5 activated the AEgain will be about1.0, so the white of the paper will receive a 100% signal amplitude.

2. The detection of ‘peakwhite’ should occur below the ‘whiteclip’ levelof the image sensor. This procedure may be incorporated for an AEcontrol with DRC. No time constants have been applied to the softwaresimulations of the AE control loop.

APPENDIX: A Simplified RGB reconstruction for a DRC in front

FIG. 14 shows a simplified reconstruction for the parallel AEmeasurement if an analog DRC has been applied in the front. The G2 pixelis regarded as the present pixel offered by the sensor. The previous redpixel has passed through a pixel delay and will be available at the sametime as G2. The G1 pixel of the previous row is matched in time with G2via a row and a pixel delay. The G1 and G2 pixel are combined into asingle green pixel. The blue pixel is also matched in time with G2 viathe row delay. Three parallel RGB signals are now available when the G2pixel is present, but only for even rows and even columns. By means of asample and hold at half the speed of the pixel clock not shown in FIG.14, a continuous RGB signal can be realized for even rows. For odd rows,no RGB signal is generated. As shown in FIG. 15, the AE measurement onlyoccurs during the even rows. By means of numerous switches between delayelements as a function of a blue pixel present in an odd row, it ispossible to realize a continuous RGB signal in odd rows as well. For theAE measurement this is however superfluous.

The above-mentioned simplified RGB reconstruction can be applied to CCDas well as to CMOS sensors. At the cost of extra row delays, notexplained here, it will of course be possible to realize a continuousmeasurement signal at a quarter of the sensor clock speed. A respectivecontinuous measurement signal is shown in FIG. 16.

1. Method of signal reconstruction comprising a dynamic range controlprocessing of an input signal of an image to generate an output signalof the image, the method comprising the steps of: providing the inputsignal; determining an amount by: specifying an input range of the inputsignal, and specifying an output range of the output signal, selecting aconvex function as a non-linear transfer characteristic capable ofcompressing the input signal according to the amount of dynamic rangecontrol processing; processing the input signal wherein the input signalis transferred by means of the convex function; generating the outputsignal as a result of the processing.
 2. The method according to claim1, characterized in that at least a peak value and/or an exposureaverage value taken from the signal is used to determine the input rangeand/or the output range, in particular taken by measurement and/orhistogram analysis of the signal, in particular taken from a luminancesignal.
 3. The method according to claim 1, characterized in that theinput signal is compressed if a peak value of the input signal exceedsthe output range.
 4. The method as claimed in claim 1, characterized inthat the input signal is compressed with regard to a mere fraction ofthe image.
 5. The method as claimed in claim 1, characterized in thatthe convex function is selected depending on the input range and/or theoutput range.
 6. The method as claimed in claim 1, characterized in thatthe convex function is formed by at least a first and a second parthaving a kneepoint as a point of intersection of the first and thesecond part wherein the first part of the convex function has an averagesteepness exceeding the average steepness of the second part.
 7. Themethod as claimed in claim 6, characterized in that the kneepoint islocated on the convex function at a specified kneelevel separating thefirst part and the second part.
 8. The method as claimed in claim 6,characterized in that each of the first and the second part of theconvex function is formed by a linear function having a constantsteepness.
 9. The method as claimed in claim 6, characterized in thatthe convex function is selected by varying the steepness of the secondpart, in particular by simultaneously keeping the kneelevel constant.10. The method as claimed in claim 6, characterized in that the convexfunction is selected by varying the kneelevel of the convex function, inparticular by simultaneously keeping the steepness of the second partconstant.
 11. The method as claimed in claim 6, characterized in thatthe convex function is selected depending on the input and/or the outputrange, wherein a combination of varying the steepness and varying thekneelevel is available.
 12. The method as claimed in claim 6,characterized in that varying the steepness of the second part isselected if the input range of the input signal exceeds a pre-determinedthreshold level.
 13. The method as claimed in claim 1, characterized inthat the image signal comprises a number of components, in particular aluminance component and/or one or more color components.
 14. The methodas claimed in claim 13, characterized in that the image signal is formedby a Y-UV-signal or an RGB-signal.
 15. The method as claimed in claim 1,characterized in that the amount of dynamic range control processing isdetermined on a Y-signal, in particular a Y-signal derived from an R-,G- and B-component or determined on at least one component of an R-, G-or B-component.
 16. The method as claimed in claim 1, characterized inthat the input signal is a digital signal.
 17. The method as claimed inclaim 16, characterized in that the digital signal is received from awhite signal balancing module and, in particular, the output signal isapplied to a gamma-control module.
 18. The method as claimed in claim16, characterized in that an amount of compression range is commonlyapplied to all components of the image signal for dynamic range controlprocessing and/or the components are processed by means of a convexfunction common to all components of the image signal.
 19. The method asclaimed in claim 1, characterized in that the input signal is an analogsignal.
 20. The method as claimed in claim 1, characterized in that theinput signal is received from a sensor, in particular a sensor matrixand, in particular, the output signal is applied to an analog digitalconverter.
 21. The method as claimed in claim 1, characterized in thatat least one of the components of the image signal is processed bytransferring the at least one component by means of a specific convexfunction according to a pre-determined amount of dynamic range controlprocessing, which has been determined specifically for the at least onecomponent.
 22. The method as claimed in claim 1, characterized in thatthe steepness, and/or the kneelevel and/or the input range is determinedfrom a specific signal component, in particular a luminance signal, andis selected for all signal components.
 23. The method as claimed inclaim 1, characterized in that the steepness, and/or the kneeleveland/or the input range is selected according to a sensor matrix and/or atemperature value of the image for each component of the signal, inparticular for a color component.
 24. The method as claimed in claim 1,characterized in that the input range and/or the output range isdetermined from a digital signal.
 25. The method as claimed in claim 1,characterized in that an exposure measurement is provided in a loop inparallel with the dynamic range control processing.
 26. The method asclaimed in claim 1, characterized in that a white balance control isprovided in a loop in parallel with the dynamic range controlprocessing.
 27. The method as claimed in claim 25, characterized in thatoriginal data of the input signal are retrieved and the original dataare provided to an exposure measurement and a white balance control. 28.The method as claimed in claim 27, characterized in that the originaldata of the input signal are retrieved by means of an inverse non-lineartransfer characteristic.
 29. The method as claimed in claim 27,characterized in that the exposure measurement is controlled to assignthe maximum output signal amplitude to a peak value of white. 30.Imaging device for signal reconstruction comprising a means for dynamicrange control processing of an input image signal to generate an outputimage signal, the image device comprising: an input means for providingan input signal; a means for determining an amount comprising: a meansfor specifying an input range of the input signal, and a means forspecifying an output range of the output signal; a computing means forselecting a convex function as a non-linear transfer characteristiccapable of compressing the input signal according to the amount ofdynamic range control processing; a processing means for transferringthe input signal by means of the convex function; an output means forgenerating the output signal from the signal received by the processingmeans.
 31. Computer program product storable on a medium readable by acomputer system, comprising a software code section, which induces thecomputer system to execute the method as claimed in claim 1 when theproduct is executed on the computer system.
 32. The computer programproduct as claimed in claim 31 comprising a module for calculation of adynamic look-up table for selection of a convex function as a non-lineartransfer characteristic depending on at least one of the parametersselected from the group consisting of: peak value, exposure averagevalue, input range, output range and temperature value.
 33. The computerprogram product as claimed in claim 31, characterized by a module forcalculating an inverse dynamic look-up table as an inverse non-lineartransfer characteristic.
 34. The computer program product as claimed inclaim 31, characterized by a module for calculating a dynamic look-uptable and/or an inverse dynamic look-up table if the input signal is ananalog signal and which is specifically adapted for at least onecomponent of the input signal.